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Abstract 

We present two symmetric function operators ff| and fff that have the property 
HfH {2 a lb) [X;q,t} = fl^nb) [X; q, t] and Hf H( 2 a lb) [X; q,t] = H^ 2 a lb) [X;q,t}. These 
operators are generalizations of the analogous operator H 2 and also have expressions in 
terms of Hall-Littlewood vertex operators. We also discuss statistics, a^(T) and b^(T), 
on standard tableaux such that the q, t Kostka polynomials are given by the sum over 
standard tableaux of shape A, Kx^q^t) = ^2, T t a ^ T \ b ^ T ^ for the case when when [i 
is two columns or of the form (32 a l fe ) or (42 a l b ). This provides proof of the positivity 
of the (q, t)-Kostka coefficients in the previously unknown cases of K x ^ 32 aib^(q,t) and 
K\(42 a i b )(li t)- The vertex operator formulas are used to give formulas for generat- 
ing functions for classes of standard tableaux that generalize the case when [i is two 
columns. 



1 Introduction and Notation 

A partition A is a weakly decreasing sequence of non-negative integers with Ai > A2 > • • • > 
Afc > 0. The length /(A) of the partition is the largest % such that \ > 0. The partition A is 
a partition of n if Ai + A2 + • ■ • + A/(a) = n. Young diagrams will be drawn using the French 
notation with the longest row on the bottom and will be identified with the partition itself 
by referring to a partition as a collection of cells. 
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For every partition A there is a corresponding conjugate partition denoted by A' where 
A- = the number of cells in the i th column of A. The arm of a cell s in p will be denoted by 
a M (s) and is the number of cells that lie to the east of s in p. The leg, / M (s), is the number 
of cells in p that are strictly north. 

A skew partition is denoted by A///, where it is assumed that p C A, and represents 
the cells that are in A but are not in p. A skew partition A//i is said to be a horizontal 
strip if there is at most one cell in each column. Denote the class of horizontal strips of size 
k by Hk so that the notation X/p E TCk means that \/p is a horizontal strip with k cells. 
Similarly, the class of vertical strips (skew partitions with at most one cell in each row) will 
be denoted by 14 . 

If A is a partition, then let A r denote the partition with the first row removed, that is 
A r = (A2, A3, . . . , A;(a)). Let A c denote the partition with the first column removed, so that 
A c = (Ai — 1, A2 — 1, . . . , Xi(\) — 1). This allows us to define the border of a partition p to 
be the skew partition p/p rc . Define the /c-snake of a partition p to be the k bottom most 
right hand cells of the border of p (the choice of the word "snake" is supposed to suggest the 
cells that slink from the bottom of the partition up along the right hand edge). We use the 
symbol htk(p) to denote the height of the /c-snake. The symbol p\k will be used to represent 
a partition with the /c-snake removed with the understanding that if removing the /c-snake 
does not leave a partition that this symbol is undefined. 

Examples 
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1) A- = (3, 1, 1) 



AJ 4 = (3, 3, 2, 2, 1) \\ 5 = undefined 



If the shape of p = \\k is given and the height of the /c-snake is specified then A 
can be recovered (A is determined from p by adding a /c-snake of height h). This is because 
A = (p h + k-h+l,p 1 + l,p 2 + l, ■ ■ ■ ,ph-i + l,Ph+i,Ph+2, ■ ■ ■ ,Pi( P ))- This will be a partition 
as long as k is sufficiently large. 
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We will consider the ring of symmetric functions in an infinite number of variables 
as a subring of Q[xi, x%, ■ ■ ■]■ A more precise construction of this ring can be found in ^H] 
section 1.2 (and roughly, the notation of this reference will be followed). 

The Macdonald integral basis JH] fo r the symmetric functions are defined by the 
following two conditions 

a) Jx = - q ax(s) t lxis)+1 )sx + J2 S ^M 

(s G A means run over all cells s in A) 

b) (Jx, J,) qt = for A ^ n 

where (, ) t denotes the scalar product of symmetric functions defined on the power symmetric 

functions by (px,Pn) qt = Sx^Zx Y\k i z \ l ^ ^ ne s i ze °f the stablizer of the permutations 

of cycle structure A and 5 xy = 1 if x = y and otherwise). The coefficients c^q, t) are 
determined by these two conditions and are rational functions in q and t. 

Also define an additional scalar product (px,P^) t = fix^zx Ylk jzp^- The (g, t)-Kostka 
coefficients are then given by the expression Kx^q^) = {J^X; q,t], sx [X]) r 

We will also refer to the basis i7 M [X;g, t] = J2x K*h(Qi t) s x[X] that is of interest in 
this paper as Macdonald symmetric functions. The H basis is related to the J basis by a 
plethystic transformation. Define the basis H^X^t] = H^[X;0,t] as Hall-Littlewood sym- 
metric functions. The H^X; t] are a transformed version of the Hall-Littlewood polynomials 
defined in [TH] and are analogous to the H^X; q,t}. 

We will use the notation of f 1 - to denote the adjoint to multiplication for a symmetric 
function / with respect to the standard inner product. Therefore (f ± g,h') = (g,fh). Note 
that hk and act on the Schur function basis with the formulas 

fi/xev k fj./xeH k 

A Schur symmetric function vertex operator is due to Bernstein [T9"l p. 95-6] and is 
given by the formula 

S m = Y J (-±) k h m+k ei (1.1) 

fc>0 

It has the property that for m > S m s^[X\ = S( m>IJl )[X]. 
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There is a Hall-Littlewood symmetric function vertex operator 

Hi = t j (-lYh m+i+j [X]eihf = J2t k S m+k hi (1.2) 

i,j>0 k>0 

with the property that for m > \i\ that H^H^X; t] = H( m ,p)[X; t) which is due to Jing ( 9J, 
P-9J). This symmetric function operator can be used to prove the existence of statistics 
on column strict tableaux such that H^[X; t] = t c ^ T > s\(T)[X]. The action of this operator 
on the Schur function basis can be expressed as follows J2H|. 

Proposition 1.1 Let X be a partition of n, let m be a non-negative integer and let k be any 
non-negative integer such that m + k > Xi, then 

H mSx [X]= Yl (-^"M-W^s^X] (1.3) 

with the understanding that if fi\ ^ is not defined then there is no contribution from that term. 

The object of this research is to find an analogous operator to Hi for the Macdonald 
symmetric functions H^[X;q,t), H&, and use it to derive statistics on standard tableaux 
that count the terms in H^X; q, t}. 

Introduce the notation for the operator that acts on symmetric functions of homoge- 
neous degree n with the formula 

Hi = uoHluR 1 = Y t n - j (-iye m+i+j [X]hief (1.4) 

where R l is an operator with the property that R t P[X] = t n P[X] for P[X] a homogeneous 
polynomial of degree n. In we show that 

Theorem 1.2 The operator 

H? = H t 2 + qH t 2 
has the property that H 2 Hr 2 aib)[X; q,t] = H^ a + 1 i b )[X; q,t\. 

In addition we used this operator to show that there exists statistics on standard 
tableaux a M (T) and b^(T) for /i = (2 a l b ) such that 

H^X-^t] = ^g b " (T) ^ (T) S A(T)[X] (1.5) 

T 
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where the sum is over standard tableaux of size 2a + b. 

This case was already considered by Susanna Fischel in [Hj where it was shown that 
the q, t-Kostka coefficients are a sum over a subclass of rigged configurations [TT] , [T2] . There 
exists a bijection between rigged configurations and standard tableaux, but the isomorphism 
between the two sets of objects is not trivial [S], [H], [H]. This case was also considered 
in PHI using a similar approach but with a different vertex operator for the Macdonald 
polynomials. 

The problem becomes more difficult when m = 3 and we will present here a formula for 
the vertex operator that adds a row of size 3, but only works when it acts on the Macdonald 
polynomials indexed by less than or equal two columns. 

Theorem 1.3 The operator 

Hf = H* + {e x [X]H\ - H\)q + ( ei [X]tf< - #<)g 2 + Stf 
= (l-q)(H t 3 -q 2 H t 3 ) + qe 1 [X]Hf 

has the property that HfHt 2al b\ [X] q,t] = Hr 32 a i h ) [X] 

We will also present an operator that adds a row of size 4 but again has the property 
that it only works when it acts on Macdonald polynomials indexed by less than or equal two 
columns. 

Theorem 1.4 The operator 

Hf = H\ + (/n[X]#| - H\)q + {h 2 [X]Hl - H\)q 2 + {e 2 [X]H t 2 - e x H\ + H\)q z 

+ {h 2 [X\H\ - h 1 H t 3 + Hl)q 3 + {e 2 [X\Hl - i^)g 4 + (e 1 [X]H t 3 - Hl)q 5 + H^q 6 
= (l-q)(l- q 2)(Hl + q 3 Hl) - q(l - q^e^ - q 2 Bl) + q\h 2 + qe 2 ){E\ + qH*) 
= (1 - q)(l - q 2 ){H\ + q 3 Hl) + q(l + q)e 1 Hf - q 2 {e 2 + qh 2 )H$ 

has the property that HfHt 2 aib\[X] q,t] = Hu 2 axb\[X] q,t\. 

With the three operators Hf , Hf and Hf acting on Hib[X; q, t] all of the Macdonald 
polynomials up to n = 8 except (3, 3, 2) are quickly computable with a computer algebra 
package such as Maple with John Stembridge's £ SF' package or Sebastien Veigneau's package 
'ACE.' 

These two formulas were arrived at by mix of chance, educated guessing, and computer 
experimentation. The proofs of these operators are not very elegant, but are also not very 
difficult to follow and only involve repeated applications of the Macdonald Pieri formula 
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They are nice because they are expressed in terms of the Hall-Littlewood vertex operators and 
the action of these operators is well understood j2S] • If a formula for the general Macdonald 
vertex operator of this sort exists, it will be of the same flavor as these but it will generalize 
the Hall-Littlewood vertex rather than have an expression in terms of it. 

In this paper, we will present statistics on standard tableaux that will show that the 
Macdonald polynomials of the form Hr 32 °-i b )[X; q, t] and Hu 2 a,ib\[X;q,t], when expanded in 
terms of Schur symmetric functions, have coefficients that are polynomials in q and t with 
non-negative integer coefficients (symmetric functions with this property will be called Schur 
positive). 



2 Formulas for H, 32ai b\[X] q, t] and Ha 2ai b\[X] q, t 

An expansion for the Macdonald polynomials Hi 2 axb\[X] q, t] in terms of the Hall-Littlewood 
symmetric functions was given in [201 • The coefficients there are factorable and of a nice 
form. Theorem 1.1 in [20] is the following result 



H^u^lX; q,t] = y^ j q a \qt a+b , i ' H^ 2il b+2a-2^[X;t] (2.1) 



{t;t)i{t-t) a . 

y] Cf'^ H^ lb +2a-2^ [X] t] 



i=0 
a 



i=0 



where (a; x)k = (1 — a)(l — ax) •••(! — ax k ~ 1 ). 

Using a translation of the Pieri formula for Macdonald polynomials p. 340, 
eq. 6.24(iv)] onto the H^[X; q,t] basis and that fi is two columns wide, we can say that 

ei[X]H(2*+iib)[X; q, t] = AH^ 2 a]b^ [X; q, t) + BH( 2a +2 1 b-i* ) [X; q, t] + CH^ 2 a+iib+i^[X; q, t] 

(2.2) 



where 



A (i-t Q+1 )(i-gr +fc+1 ) 

(l_ g 2 t a+6+l)( 1 - qt *+l) 1 

(1 - qt b )(l - qt** 1 ) ^ ' 1 

c _ (l-g)(l-^) (25) 

(1 - qt b )(l - qH^ 1 ) 1 ' 
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Rearranging this formula to solve for Hr 32 aib^[X; q, t) gives 

H(32 a i b )[X; q, t] = A' e\[X]H^ib)[X] q, t] — B H^ 2 a+2]b-i)[X; q, t] —C H^ 2 a + 1 i b + 1 )[X; q,t] 

(2.6) 

where 

A , = (l-g 2 t a+b+1 )(l-< +1 ) 

(1 -t a + 1 )(l-qt a + b + 1 ) 1 ' ' 

R , _ (l-q 2 t a+b+1 )(l-t b )(l-q) 

(1 _^+l)(l -qt b )(l 1 ' J 

r (l-g 2 t fc )(l-gt a+1 )(l-g) 

The last two terms on the right hand side of equation ()2.6|) can be expanded in terms 
of Hall-Littlewood symmetric functions using (|2.1|) . The first term on the right hand side 
of the equation can be expanded in terms of Hall-Littlewood symmetric functions by using 
(|2.1j) followed by formula ()2.2)1 with q = we have that 

ei[X]# (2 .iv)[X;t] = # (2 *i,+i)[X;t] + (1 - ^)F (2 , + i ls -i)[X; t] + (1 - t x )H (32 *-i lv) [X; t\ 

(2.10) 

Therefore, arriving at a formula for Ht 32 a\b\[X\ q,t] is a matter of applying formulas 
(j2.1j) and (|2.1()jl to (|2.fi|) . Unfortunately, the result is not nearly as nice as it was in the case 

of fl2U). 

Proposition 2.1 

a 

H( 32 «ib)[X;q,t] = ^c- a ' b) (l - q 2 t a+b+1 )(l - qt a+1 )H( 32 i 1 2a+b-2 i )[X; t] + q a+3 H^2a+b+s^[X; t] 

(1_ q H a+b+1 )(l -qt a+1 ) 



i=0 
a+2 



a-t-<s / 

i— i \ 



(1 -^-^(l - gt a + 6 + 1 - i ) 



i=i 

(1 - g 2 ^ +6+1 )(l - gr +1 )(l - f )(1 - t 2a + b+A ~ 2i ) 
+ q (l- qt a+l+b - l )(l - t a+1 - l )(l - t a+2 - l )(l - g t a + 2 + 6 - i ) ^ ' ^ 

(1 _ g2 t a+b+l)(l - t h )(l -q)(l- T+ 2 ) 

9 (1 - t a+2 ^)(l - qt a + l+b -*)(l - t a+1 ~ l )(l - qt b ) 
(1 - g 2 t fe )(l - qt a+1 )(l -q)(l- qt a+b+2 ) 
~ (1 - qt a+b+2 - i )(l - qt a+1+b - i )(l - t a + 1 ~ < )(l - ?t 6 ) 
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A slightly simpler expression for this result can be given after the vertex operator is 
introduced, but it is not obvious when the coefficients of Proposition 12.11 are in a reduced 
form. 

The proof of Proposition ^, ll uses the following lemmas that each follows from a simple 
calculation from the definition of q = g a ~ t (gt a+b ~ t+1 ; t)i i t . t yu.^ — ;■ 



Lemma 2.2 For < z < x 



(l - qt x +y- z ) 
z+1 q {l - t x - z )(i - qt x +y~ z ) 

(1 - qt x+ v- z )(l - t x ~ z ) 



(2.12) 
(2.13) 
(2.14) 



There are no tricks involved in the reduction of (|2.fi|) to Proposition ^. 1| just algebraic 
manipulation. Hence we leave the details of the proof to the reader who may be able to 
discover a better expression. This proposition is necessary only for comparison to a similar 
expression for if|*i^ 2 a i b )[A; q, t\. o 

To derive a formula for H^ 2 a i b )[^'i Q->t] we will use the same brute force method for 
finding equations for the coefficients of the Hall-Littlewood symmetric functions. If we add 
a horizontal strip of size 2 on a two column Macdonald summetric function by multiplying 
by g2[X;q,0] where g r [X;q, t] is defined in [THl eq. 2.8, p. 311] then we have the following 
terms 

g 2 [X; q,Q]H(2*+ii<>)[X; q,t] =AH^+^i b )[X; q,t] + BH^+n b ~ 1 )[X; q, t] + 

CH (32alb+1) [X;q,t] + DH {42alb) [X;q,t} (2.15) 

where A, B, C, and D are given as 
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A (l-g 2 ^+ b + 1 )(l-gt a + 1 ) (2 ' 16) 
(l-t b )(l-qt^) 
(l-q)(l-q 2 t a + 1 )(l-qt b )(l-q 2 t a + b + 1 ) l ' ' 

(1 -t a+1 )(l -q 2 t b ) 

(1 -q)(l- qt b ){l - g 3 i a + b+1 )(i - qt^ 1 ) 

(1-^)(1-^) 
(1 - g)(l - g 2 )(l - g^+^JCl - g 2 t Q+1 ) 1 ' ' 

Rearranging terms, we have that 

H(42 a i b ) [X] q, t] =A'H(2 a + 2 i b ) [X] <7> t] + B H(32a+iib-i) [X; q, t] + 

C'H {32alb+1) [X; q, t] + D'g 2 [X; q, 0]H {2 * +nb) [X; q, t] (2.20) 

where A',B',C, and D' are given as 

A , = (l-g)(l-g 2 )(l-g 3 t Q+fe+1 )(l-g 2 t a+1 ) 

(l-q 2 t a + b + 1 )(l-qt a + 1 )(l-t a + 1 )(l-qt a+b + 1 ) K ' 

r>/ _ (l-t")(l-g 2 )(l-g 3 r +fe+1 ) 

(1 -qt b )(l -q 2 t a+b + r )(l -t Q+1 ) 1 J 

r , (i-g¥Q(i-g 2 )(i-g 2 r +1 ) 

(l-gt f) )(l-gt a + 1 )(l-gt a+6 + 1 ) 1 J 

D , = (i-g)(i-g 2 )(i-^ Q+fe+1 )(i-g 2 t a+1 ) 

(1 -*•+!) (1 - gt«+6+!) 1 ' ' 

Only the last term in this expression does not have an expression in terms of Hall- 
Littlewood symmetric functions yet. As in the (32 a l 6 ) case, this can be computed by trans- 
lating [121 p. 340, eq. 6.24(i)] and setting q = 0. We have that 

h 2 [X]H^iy)[X;t] = iif(2*+iiv) + (1 — t y )H( 32 x lv -i) [X; t] + 

(1 -r)# ( 32*-ii»+i)[X;t] + (1 -t x )H (i2 *-i ly) [X;t} (2.25) 
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and 



and 



hi [X\ 2 H(2x\y) [X; t] =if(2*iH-2) [X; t] 

+ (2-t y+1 -t v )H {2 *+i lv) [X-t\ 

+ (1 - t v )(l - ^- 1 )iJ (2 , + 2 1!/ - 2) [X; t\ 

+ 2(1 — t x )H( 32 *- 1 iy+ 1 ) [X; t] 

+ (l-^)(2-r+ 1 -^)F (32n9 - 1) [X;t] 

+ (l-t-)(l-t- 1 )ff ( 3 32X _ 2ll/) [X;t] 

+ (1 - t x ) (1 - *)iJ (4a -i lv) [X; t] (2.26) 



^2[X;g,0] = (1 _ g) ( 1 _ g2) ^W + Y37^[X] (2.27) 



The next proposition follows by taking coefficients in the formulas above. 

D' 

(l-q)(l-<? 2 ) 



Proposition 2.3 Let E' = , x _ P, 1 _ 2 , and denote the coefficient of H( 32 n 2a + b - 2i )[X', t] in 



Hr 32 aib)[X; q, t] (as given in Proposition \2.1\) by d^' and the coefficient of H^ 2 i r i + 2a + b - 2t )[X', t] 

in i7( 32 aii>) [X; q, t] by e\ a ' b ^ . The coefficients for the symmetric function if M [X; t] in the Mac- 
donald symmetric function Hu 2a ,xb\[X\ q,t] are given by the following table of expressions. 



Ii = (2 l l 4+2a+b ~ 2i ) A'c^ +2 ' b) + B'et +1 ' b - 1] + C'et M1) + E'{1 - q)cf^ b) + (2.28 

qE'( c { a+1 > b ) + (2 — t 2a + b ~ 2i + 5 — f2a+b-2i+4^ c (a+l,b) 
2a+6-2i+6 \ ( i + 2a+b~2i+b \ (a+1,6) \ 



+ (I - t 2a+b - 2i+& )(\ - t 



C: 



i-2 



fi = ^2 l l 1+2a+b - 21 ) S'4 a+1,6_1) + C'd { t b+1) + (2.29) 

E '(l - q)((l - ^+6+2-2^+1,6) + ^+1^(0+1,6)^ + 

qE' (2(1 - f +1 )c£j 1,6) + (1 - t 2a+b+2 - 2l )(2 - t i+1 - t )cf +l ' b) ) 

^ = ( 33Tl 2a+b-2i-2^ gE f^ _ f +2^ _ f+ ly{a+l,b) ^ ^ 

H = ( A2 l l 2a+b - 21 ) E'(l - - qt)c^ 1,b) (2.31) 

where if j < then = and the coefficients c[ a ' b \ A', B' , C and D' are all given in 
the text above. For all \i that do not follow a pattern in this table, the coefficient of H^[X;t] 
in Hu 2al b\[X] q,t] is zero. 
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Although using this technique for computing expansions of Macdonald polynomials 
might be used with other bases, here it seems that a general formula for the expansion of 
Hn\X\q,i\ in terms of H^[X;t] will not be useful since patterns in the coefficients do not 
seem to exist as they did in the case when [i = (2 a l b ). We will show that these two particular 
expansions do have a use to make the computation of Macdonald polynomials in these special 
cases much easier. 

3 The Vertex operator Hf 

Our purpose in giving an expression for H( 32 a i b )[X] q,t] and Hu 2al b\[X] q,t], is to show that 
the expressions given in Theorems II. HI and 11.41 have the properties HfH(oaib\[X; q, t] = 
H( 32 *i>>)[X;q,t} and H 4 H^b^X] q, t] = H^ 2al b^[X;q,t\. 

The vertex operators Hf and Hf are given as expressions using the H^ and H l m 
operators and so we will need the action of these operators on the Hall-Littlewood basis. 

Lemma 3.1 

H\H(2^\v)\X] t] = t x H^w+3) [X] t] — t x+v+ (1 + t)H(2*+iiy+i)[X; t] 

_ t x+ y + \l - t«) J ff (2 . + 2iv-i ) [X; t] + t 2x+y+1 H^ lv) [X- t] 

Proof: We note the following three commutation relations: 

frt rrt A.m—1 Tjt fjt (o i \ 

Tjt Tjt J Tjt Tjt , I Tjt Tjt Tjt Tjt (O Q\ 

n m-\ n n — ln m n n-\ ' LrL n rL m-l n n-\ n m K -^) 
H L Ht m+l = tH m+X Ht m ( 3 - 3 ) 

A proof of ()3.2|) is in [T§1 p. 238] and ()3.3|) is a specialization of that. The proof of 
equation ()3.1|) is the same as [221 Lemma 2.4]. 
For ()3.2|) . in particular we have that 

H\H\ = tH\R\-(\-t)RlRl 

And HsH(2=civ)[X;t\ can be computed by calculating commutation relations. 

H 3 H( 2 *iy)[X; t] = t x {H l 2 ) HlH^v)[X; t] 

= t x (Hty(Hiymi 

= t x (HQ* (Hl) y (H (13) [X; t]-(t + t 2 )H (21) [X; t] + t 2 H (3) [X; t}) 
= t x H(2 x iv+ 3 ) [A; t] — t x {t + t 2 )t y+1 H^+^iv+i) [A; t] 

- t x+2 t v -\l - t^H^^y-^X; t] + t^f+^H^^iX; t] 
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o 



We now give an outline of the proof of Theorem II. HI 

Proof: We use equation (j2.1j) and Lemma ETT1 and show that the expression is equivalent 
to Proposition 12.11 

HfH {2alh) [X-q,t] ={l-q)H t 3 H (2alb) [X;q,t]+qe 1 [X]H {2a+llb) [X;q,t] 
+ q 2 (q-l)H t 3 H {2alb) [X;q,t] 

a 

=(! - q) cf^H^^a+b^lX] t] 

i=0 

a+1 

+ 1 ^ c ( i a+1 '^ei[X}H^ 2 i 1 2a+2+b-2i^[X; t] 

i=0 

a 
i=0 

— ^2a+b -{- t)H( 2 i + ll2a + b-2i+lj[X]t\ 

— t — t ) £1 ( 2 i + 2-l2a + b-2i-l\ ) \Ji. , t\ 

From here it is only algebraic manipulation to reduce the expression to one like 

a 

= 4 a ' fe) (l - q 2 t a+b+l )(l - < +1 )# (32W 6- 2i) [X; 1] + q a+3 H {1 2 a+b+:i) [X; t] 

+ E (^ (a+1,6) + (! - * 2a+b+4 ~ 2i )<?c!-t 1,b) (3.4) 

8=1 

+ g 2 (g - l)cf ' 6 V - g 2 (g - l^f^^^l + t) 

-q 2 (q - l)c£$1? a+b+ *- i (l - t 2a+b+4 ^)) H {2il2a+b+3 - 2i) [X; t] 

by converting all coefficients in terms of cf using equations ()2.12j) . (|2.1Hjl and ()2.14|) we 
derive the following expression 
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^ ] cf 1 '^ (1 — g 2 t a+6+1 )(l — gt a+1 )if(32ii2a+i,-2i)[X; t] + q a+3 H^a+b+s^ [X; t] 
(1 - qt a+1+b )(l - t a+1 ) 



i=0 
a+2 



i=l 



Q 7l „i„XU/i-A/1 J.^4_l_i\/1 I^XoZaTi „i /T _l_9.a.7,_i\ (3-5) 



(l_ gi a+l+6-i)( 1 _ t a+l- 

1 - f2«+H4-2i)(l _ _ t a+1 )(i - f ) 

1 - g£<H-l+6-i)(l - - t a+2 -*)(l - qt a + 2 + b -> ' 



+ g(g - l)f - q\q - 1) (1 _ - J^ff - ^ + t) 

(1 _ t i)(l _ f -l )t 2a + 6 + 3-, (1 _ ga+6+4-2i) X 
(? -!)(!_ gt a + 6 + 2- t)(1 _ ifl+2 -i )(1 _ qt « + ^-*)(l - ta+l-i) J ^(2H™- 2l) [X; tj 

which can be shown to be equivalent to Proposition 12. II bv more algebraic manipula- 
tion or by appealing to a computer algebra package such as Mathematica. o 



As a corollary, ()3.4j) is an expression for the expansion of /f^ai&JX; q, t] in terms 
of the basis H^X't] since it seems to be a slightly nicer expression for the Macdonald 
polynomials than either (j2.11|) or (|3.5p . 

We make the following substitution of notation to indicate what the pieces of this 
operator represent. Define the following operators 



fl™ =H\ ifffla =e 1 H t 2 -H t 3 

f/ffla =e 1 E\-E\ Hm = H\ 

Then notice that iff* has the expression 

iZf = + giA + q 2 H^ + q 3 lM (3.7) 

Suddenly the operator that has so far been very unelegant, looks like it is shaping 
up. In the following sections we will show that not only is HfHi 2al b\[X\ q, t] Schur posi- 
tive, but so are all of the pieces of this expression (H^^H^a^lX; q, t), H^Hr 2al b\[X;q,t), 

H^H^ 2 aib)[X; q,t], and iMlI^a^lX; q,t\) and that each is a generating function for a sub- 
class of standard tableaux. 
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4 The Vertex operator Hi 



As with the vertex operator H^, we will use what we know about the action of H l m and 
on the Hall-Littlewood basis to compute the action of (1 — q)(l — q 2 )(H\ + q 3 Hl) + q(l + 
q)e\Hf — q 2 (e2 + qh 2 )Hf on equation (|2.1)1 . The only piece of this equation that we have 
not already given an expression for is the action of H\ on the Hall-Littlewood symmetric 
functions. 

Lemma 4.1 For a,b > we have that 

H^H( 2 a\ b \\X\ t\ =t a H(2a-\b+i) [X ] t] — t^^^ (1 + t + t 2 ) H( 2 a+ 1 ;[6+2 ) [ X \ t\ 

- t a+b+1 (l + t-t b - t b+1 - t b+2 )H (2 a+2 lb) [X; t] 

-t a+b+1 (l-t b - 1 )(l-t b )H (2 a + 3 lb -2 ) [X;t} (4.1) 
+ t 2a+b+2 (l + t)H {32alb+1) [X; t] + t 2a+b+2 (l + t)(l - t 6 )^ (32a+llb -i)[X; t] 

, j.2a+Jj+2 / 1 + a\ tt f V- +1 j-2a+fe+3 rx 

+ t (1 — t Jii(33 2 a-1 1 6[A, tj — t _H( 42a ii>) 

Proof: First set a = 0, and prove that for all b greater than or equal to zero we have that 
the theorem is true by induction using the identities (jH.lj) . (|«LHj) . and (|H.2jl in the special 
cases of H\H\ = tH\H\, E\E\ = tH\H\ + (t 2 - = ti^i^l + (t- \)E\R\, and 

H\H\ = tH^Hl. The base case is a calculation of 

^(^[x^-^i+t+t 2 )//^ 

Now for a > we have that 

H^iif(2 a i i ')['^"j = tH^H^Hf^a-iib-^X^t] 
Using the relations E\H\ = tH\H\ and = H\H\ + (t - l)H\Hl, the result follows, o 

We are now ready to prove the vertex operator property. 

Proof: (of Theorem EU) 

For each /x that appears in Proposition I2.HI we take the coefficient of H^[X;t] in the 
expression 

HfH {2alb) [X; q,t)=(l-q)(l- q 2 ){H\ + q 3 Hl)H (2alb) [X; q, t] + q(l + q)e 1 [X)H {32alb) [X; q, t] 
- q 2 (hl[X] + {q- l)h 2 [X])H (2a+llb) [X; q,t] (4.2) 
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We need Proposition EH equations fl2HJ), (pHTIjl . fTZty , (pOKjl and flUEJ), and one more 
translation of the Macdonald-Pieri rule. 



ei[X)H( 32 "ii>)[X;t) —H^ 2al b+^[X\t] + (1 - t b )H( 32 a + 1 i b - 1 )[X; t) 

+ (1 - t a )^ ( 332-ii^)[X; *] + (1 - t)H {42alb) [X; t] (4.3) 

As in Proposition ^. 31 denote the coefficient of H^i 2a + b -^)[X; t] in Ht 32 aib)[X; g, t] (as 
given in Proposition 12. 1|) by e^ a '^ and the coefficient of if( 2 ii3+2a+6-2 l )[X; £] in f/"/ 32 a 1 6)[X; g, t] 
bye] '. 

If fi= (42 l l 2a+b - 2i ), the coefficient is 
(1 - g)(l - g 2 )(l - t 2 ^+ 3 g 3 )c>' 6) + g(l + g)(l - t)d™ + g 2 (l - t i+x )(t - q)c^ (4.4) 
If n = (332 i l 2a+6 - 2 - 2i ), then the coefficient is 

(1 - g)(l - g 2 )g¥^+ 2 (l - f +1 )c>f + g(l + g)(l - t i+1 )d^ - g 2 (l - f+ 2 )(l - 

(4.5) 

If fi = (32 i i 2a + b + 1 - 2i ), then the coefficient is 

(1 - g)(l - g 2 )g 3 t 2a+b+2 (l + t)(cf> b) + (1 - t 2a + b+2 ' 2i )c\ a _:f) 

+ g(l + g)((l - f +1 )e£'? + df' b) + (1 - t 2a+b+2 - 2i )dt^) (4.6) 

- g 2 (2(l - f +1 )c£f 1,6) + (1 - i 2a+6+2 - 2i )(2 - f +1 - f )c, {a+1 * fe) 

+ (g - 1)((1 - £2a+6+2-2 i)c (a+l >6 ) + (1 _ ^+1^(0+1,6)^ 

Finally, if fi = (2 i l 2a+b+4 ~ 2i ) , then the coefficient is 

(1 - g)(l - q 2 )q\t l c { t b) - t 2a+b+2 -\l + i + t 2 )ct b ? 

_ j2a+b+?j-i _|_ ^ _ ^2a+6+4-2i _ ^2a+fe+5-2i _ ^2a+b+6-2i\ (a,6) 
_ ^2a+6+4-i^ _ £ 2 a+b+5-2«^ _ ^2a+fe+6-2^ c (a,b)^ 

+ 9(1 + g)(e 4 (a,fc) + (1 - t 2a+6+5 - 2i )eh 6) ) (4-7) 

_ g 2 ( c ( a+1 ' 6 ) + (2 — £ 2a + 6 + 5 ~ 2i _ ^2o+6+4-2i\ c (a+l,6) 
-f (1 _ ^2a+6+6-2i\ _ ^2a+6+5-2i\ c (a+l,b) 



+ (g - i)Cl 



(a+l,&h 
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These can be shown to be equivalent to Proposition 12. HI by hand (with an enormous 
amount of patience) or by reducing these expressions on computer using a computer algebra 
package such as Mathematica. o 



We introduce the following notation as we did in the case of Hf to better demonstrate 
the structure of this operator. Define the operators 



//E§ _|_ jj fiba 

h\ 
h\ 



h 2 H\ 



hi 



e 2 H\ - e x H\ + H\ 



i#a = e x E\ - H\ 

#Ski = h 2 H\ - hiHl + H\ 

iM + lA = e 2 H\ - H\ 



e\H\ — H\ 



El 

3 



(4.8) 



I am unaware of how to separate the operators ijffl + i^fibm anc l ffWt + HSs, but proofs 
will work nearly as expected as long as we consider these as two single entities. Notice that 
Hf has the expression 



Hf 



-H 1 



TT2H141 



+ q 



^//El _|_ 



+ 



(4.9) 



+ q- 



HI 

+ q 6 Hm 



These operators (and the ones for Hf 



uH T u 



>i/t 



were all defined so that they have the relation 
H wT (where toT represents the diagram flipped about its diagonal). 



Using the relation H^X] q, t] = ujH^X; t, q], we note that ooH^oo, ujH^uj and ujH^uj 
add a column of size 2, 3 and 4 respectively to a Macdonald polynomial indexed by a two 
row partition. 

These vertex operator formulas provide a fast method of computation on a computer 
algebra package (such as Maple) of the Macdonald polynomials for partitions with two rows, 
two columns, or of the form (32 a l 6 ), (abl), (42 a l fc ) or (abl 2 ). Theorem PI PI and PI 
are therefore enough to calculate all of the Macdonald polynomials and the (q, t)-Kostka 
coefficients through n = 7 and all but one partition at n = 8, (3, 3, 2). 

The operators H\ , Hf and Hf can be modified so that they work on the other 
Macdonald bases. Let V be the operator with the property VJ^[X;q, t] = H^[X;q,t}. 
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Define J^* = V^H-fiV then J-£ has the corresponding property on the J^[X; q, t] basis. The 
formulas seem to take on an interesting form when expressed in this way. 

The methods presented here might be used to handle a few additional cases of deriving 
formulas for vertex operators for Macdonald polynomials, but experimental computations 
indicate that using the Hall-Littlewood vertex operators will not give nice expressions in 
general. There seems to be another family of vertex operators that has the Hall-Littlewood 
vertex operators as a special case that might be used to show that the (q, t)-Kostka coefficients 
are polynomials with non-negative integer coefficients. 

5 Tableaux, charge, and two column Macdonald poly- 
nomials 

A column strict tableau is a diagram of a partition (or skew partition) with each cell labeled 
with a positive integer such that the labels increase weakly traveling from left to right in 
the rows and the labels increase strictly traveling from bottom to top in the columns. A 
standard tableau is a column strict tableau with the numbers 1 to n where n is the size of 
the partition. 

Let T be a column strict tableau. Denote the shape of the tableau by A(T), the total 
number of cells in the diagram by \T\, and the number of cells labeled with an % by the 
symbol Tj. The content of the tableau will be the tuple fJ>(T) = (Ti,T 2 , . . . , Th) (where h 
is the highest label that appears in the tableau). T is said to be of partition content if the 
content vector /x(T) is a partition. The content of a word is defined similarly (the tuple 
consisting of (the number of l's in the word, the number of 2's in the word, etc.)) 



We begin by defining an algorithm for calculating the statistic called charge on col- 
umn strict tableaux of partition weight. This statistic was introduced by Lascoux and 
Schutzenberger. 

First, charge is defined for words of content weight /j, — l n . An index is given to each 
letter in the word. The index is assigned to 1. If the letter i has index k then the index of 
the letter i + lis&ifi + l lies to the left of % and the index is k + 1 if % + 1 lies to the right 
of i. The charge of the word is defined to be the sum of the indices. 

If w is a word with content of partition weight then it is first broken up into standard 
subwords by the following procedure. Place an x under the first 1 in the word traveling from 
right to left. Next place an x under the first 2 traveling to the left from there. Continue 
placing an x underneath each of the letters 1 through l(/i(w)) traveling from right to left and 
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beginning again at the right side of the word each time the left hand side is reached. The 
first standard subword consists of the letters that have xs underneath them read from left 
to right. Erase these letters to form a new word w' and repeat the procedure forming the 
next standard subword with the labels 1 through l(n(w')). Stop when all letters have been 
erased. The charge of the word is then defined to be the sum of the charges of the standard 
sub words. 

The reading word of a tableau is the word formed by reading the entries in the cells 
in each of the rows from left to right, starting with the top row. Denote the reading word 
of T by 1Z(T). Lastly, the charge of a tableau T is defined to be the charge of the reading 
word of T. Denote the charge of a word w (or tableau T) by c{w) (respectively c(T)). 



Example 5.1 



7 










3 


4 


6 






2 


2 


3 


5 




1 


1 


1 


2 


4 


8 



A tableau of shape X = (6,4,3,1) and content fi = (3,3,2,2,1,1,1,1). The reading 
word of this tableau is 73462235111248. The word has standard subwords 73625148, 4231 
and 12. The first standard subword has charge 6, the second charge 2, the third charge 1. 
The charge of the tableau is 9. 

If x, y and z are letters in the words u and v and w and w are subwords then say that 
two words u and v are elementary Knuth equivalent if either u = wxzyw and v = wzxyw 
where x < y < z or u = wyzxw and v = wyxzw where x < y < z. 

Next say that two words u and v are Knuth equivalent and write u ~ v if they are in 
the symmetric, transitive, reflexive closure of the elementary Knuth equivalence. 

There are several important facts about words and the charge of words that we will 
use to develop statistics here. These are well known results and proofs can be found in [THj . 

ra, 0, i- 

Fact 5.2 Every word is equivalent to the reading word of a unique tableau with partition 
shape. 

Fact 5.3 If u ~ v then c{u) = c{v). 
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Fact 5.4 If w = wil a W2 where w\ and W2 are subwords of w that do not contain 1 then 
c(w) = c((w2Wi) I 1 ) + I W2 1 where the notation of [ k (dually, ]k) indicates that the letters of 
the word to the left of this symbol have their labels decreased (increased) by k. 

Fact 5.5 If w = x\%2 ■ ■ ■ x n is a standard word, then c(w) = ( g ^ ~~ c[x n x n -\ ■ ■ -X\). 

Fact 5.6 If w — X\X2 ■ ■ -x n is a standard word and w is Knuth equivalent to the reading 
word of a standard tableau T , then x n x n -\ • • • x\ is Knuth equivalent to the reading word of 
the conjugate tableau uoT . 

The algorithm of Robinson-Schenstead will be used implicitly throughout the fol- 
lowing sections. Knowledge of the expressions row/column insertion/evacuation and their 
relations to Knuth equivalence are assumed in some of the algorithms. Frequently in the 
text we will identify a tableau with its reading word 

It is due to the development of this theory and the results of Lascoux and Schiitzenberger 
03, dl, P, that the Hall-Littlewood polynomials H^[X;t\ = Y^t^cst^^ S KT)\ X \ 
where CST^ is the collection of column strict tableaux of content fi. 

To describe the standard tableaux statisics for the case that the Macdonald polyno- 
mials were indexed by {2 a l h ) we defined two procedures for building the standard tableaux 
of size n + 2 from those of size n. The statement of these definitions will be for arbitrary m 
but for now set m = 2. 

Let T be a standard tableaux of shape A h n and let p be a partition of 2n + m such 
that p/X G 7i n+m . Consider the cells that are in A and not in p r and perform one column 
evacuation for each cell from right to left so that cells are evacuated in increasing order. Let 
the row of evacuated cells be R and the remaining tableau be T. Raise the labels of the cells 
of R and T by m and row insert the labels 1 through m followed by all of R in increasing 
order. The result will be the definition of the tableau H^T. 

We may say that if H^T and p are given, T can be recovered by evacuating 12 • • • mR 
a row of size m + |A/p r | from H^T to leave T of shape p r . Denote this reverse operation by 

UK) '• 

Also define the operator that adds a column block of m cells to be the transpose of 
the operation H^T. Again, let T be a standard tableaux of shape A h n. Let p h n + m 
be a partition with p/X G V n+m . Consider the cells that are in A and are not in p c and 
perform one row evacuation for each cell from top to bottom so that the cells are evacuated 
in increasing order. Let the column of evacuated cells be C and the remaining tableau be 
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f. Raise the labels of the cells of C and f by m and column insert the labels 1, through m 
and all of C in increasing order. The resulting tableau will be the definition of H^T. For 
the same reason as before, T can be recovered if H£ t T and p are given. 



Example 5.7 Let T = 



5 6 



2 


4 


5 


and T = 


1 


3 


6 



The labels are all raised by two and 1, 2 and R are all inserted into T to form the tableau 



W 2 T = 



3 


5 


8 




1 


2 


4 


6 7 



Consider T is the same tableau, but now p = (4, 3, l 7 
column of size 2 we have that C 



6 



and T 



2 


4 




1 


3 


5 



In the procedure for adding a 
. Therefore, 





8 








H P 2 T = 


2 


4 


6 






1 


3 


5 


7 



We also introduced the notion of the (2 a l 6 ) — type of a standard tableau. The type 
is defined recursively by 'unbuilding' the standard tableau in blocks of size 2 (for now set 
m = 2 in the following definition). 

If IH2I B is a subtableau of T then let R be the first row of T and T be T with 

the first row of T removed. Define H^T to be the tableau formed by column inserting the 

labels in R that are not the lables 1 through m into T in decreasing order then lower all of 

the labels in the result by m. 
m 

is a subtableau of T then let C be the first column of T and T be T with the first 



If 



column of T removed. Define H^T to be the tableau formed by row inserting the labels in 
C that are not a 1 through m into T in decreasing order then lower all of the labels in the 
result by m. 

Now define the (2 a l 6 ) - type of T as follows. 

• If a = and p = (l b ) then type^T) = (m 6 ). 

• If a > 1 and 2 lies to the right of 1 in T, then typei 2 a\b\(T) = (ran, type^a-ii&^H^T)) 

• If a > 1 and 2 lies just above 1 in T, then type^ib^T) = (g, iype^a-i^^HjT : T)) 
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3 






Example 5.8 LetT = 


2 


6 






1 


4 


5 



and H^H^T 



1 



Therefore, type^(T) = (0,1 ma). 

It was necessary to introduce more machinery in |22^ to prove the following results. 
Here we will take them as given without defining the handful of tableaux operators that 
were necessary to verify that they are true. The first is Proposition 3.5 and the second is 
Corollary 3.11 of [22]. 

Proposition 5.9 Let n = 2a + b. The statistic a^(T) and b^(T) where p = (2 a l 6 ) from 
equation < li.,5j) may be defined to be 

a 

a M (T) = c(T) - ^((n + 1) - 2i) X (type tl (T) i = ma) 
i=i 

a 

M T )=I>(^COi=ffl) 



i=i 



Proposition 5.10 Let n = 2a + b and T G ST n and let p be a partition of2n + 2 such that 
p/X(T) G H n+2 then 

type {2 a+i lb) (H p 2 T) = (nm,type {2 a lb) (T)) 
Similarly, if p/X(T) G V n + 2 then 

type( 2 a+i lb) (H%T) = ($,type [2 a lb) (T)) 



6 Statistics for fi = (32 a l 6 ) or fi = (42 a l 6 

When acts on a Macdonald symmetric function, we observe experimentally that if m > 
/ii— 1, then H^H^X; q, t] is Schur positive. This can be explained for the case that p = (2 a l b ) 
where we have a combinatorial interpretation for the terms. We then use this result to show 
that H s H( 2 a lb -j [X; q,t] is Schur positive when S is a standard tableaux of size 3 or 4 where 
the H s operators are given in equations ()3.6|) and (j4.8|) . 

The definition of type^ 2 a+i lb )(H. 2 T) is (en, type^a^CH^H^T)), Proposition 15.101 is 
equivalent to type^ 2 a lb - ) (ii 2 1 ii 2 T) = type^a^iT). First, extend this result for all m > 2. 

In all of the results in this section we will assume that m > 2. 
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Lemma 6.1 Let T e ST n and let p be a partition of2n + m such that p/X{T) e Ti, n+m then 

^e (2 ai6 ) (H^ 1 H^T) = type i2 a lb) (T) 
Similarly, if p/X{T) 6 V n+m t/ien 

type {2 a lb) (Yl^W m T) = type (2 a lb) (T) 

Proof: We will just consider the first case since the proof of the second is mostly a matter 
of changing to and 'row' to 'column.' 

Let p = A(H^T) r . We claim that H^H^H^T = H P 2 T. 

Let S be the tableau of shape p r formed by evacuating the cells in X(T)/p r and let 
R be the evacuated cells in the procedures for calculating H^T and H 2 T. We will analyze 
the procedures without making changes in the labels for H^T and instead of inserting the 
labels 1 through m, insert m 'spaces.' The procedure for calculating the right hand side says 
row insert two spaces and all of R into S. 

The procedure for calculating the left hand side says row insert m spaces and all of 
R into 5*. Next, row evacuate the entire first row of the result, throw away all but 2 of the 
spaces and reinsert what is left. Clearly, since row insertion and row evacuation are inverses 
of each other, this is equivalent to the procedure for calculating the right hand side. 

The result follows from a couple of applications of Proposition 15.101 since 

type {2 « lb) (H7jW m T) = type (2alb) (U 2 ^H^H^T) 

= type (2alb) (H 2 - 1 H p 2 T) (6.1) 
= type {2 a lb) (T) 

o 



Define an involution, on partitions. Let 7 = p\ n . If > 7^ then define I\{p) = 7 
with aim - snake of height h + 1 added. If Xh < 7^ then let /£(p) = 7 with an n — snake 
of height h — 1 added. Lemma 3.15 of [221 was ^ e following result. 

Lemma 6.2 1^ is an involution on partitions p such that p/X G 7i n , p\ n exists and X 7^ p\ n 
with the property that ht n (I™(p)) = ht n (p) ± 1 and p\ n = I\(p)\ n - 

Example 6.3 Let n = 10, X = (5, 5, 2), and p = (12, 5, 5), then p\ n = (4, 4, 4) and ht n (p) = 
3. Ix(p) = (13,5,4) because Ix(p) = (4,4,4) with a 10 snake of height 2 added. 
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Lemma 6.4 For a pair (T, p) forT a standard tableau, p/A(T) £ H n+m 

c(H^T) = c(T) + |A(T)/p r | + (2) + (m - l)n (6.2) 

Proof: We note that H^T = (T | m )12 • • • m(i2 |m) where T and i? are given in the 
procedure for the calculation of H^T. T = RT and \R\ = \\(T)/p r \. Apply Fact 15.41 m 
times, o 



Lemma 6.5 The following two conditions are equivalent for a pair (T, p) such that pj A(T) £ 
TCn+m and p\ n exists. 

H^H^T = T (6.3) 
\(W m T) = p\ n (6.4) 

Moreover, if(T,p) satisfies these these conditions, then ht n (p) = 1. 

Proof: Since p/A(H^T) £ 7i n then if A(H^T) = pj„ then we have that p/p\ n £ W n , 
and hence n = 1. 

Let T and i? be (respectively) the tableau of shape p r and the row of size \\(T)/p r \ 
in the procedure for H^T. 

If A(H^T) = p\ n then pj n = (pi — n, p r ) = (m + |i?|,p' r ) hence the first row of 
H^T (which must contain 12- ■ •m(R fm) since it was row inserted in increasing order) is 
12 ■ ■ ■ m(R t m ). Therefore H^H^T = RT = T. 

If H^H^T = T then the first row of H^T will be denoted by 12 • ■ ■ mR and the 
remainder of the tableau will be called T. We have that RT = T and we want to show that 
\R\ = \R\. By Lemma (6.41 and repeated application of Fact 15.41 we have that 

c(T) = c{TY2 ■ ■■mR) - \R\ - f 2 J - (m - l)n = c(T) + - \R\ (6.5) 
Therefore A(H^T) = (m+ |-R|,p r ) = p\ n . o 

Call a pair that satisfies the conditions of Lemma 16.51 stable and a pair (T, p) such 
that p/A(T) £ 7i n+m and pj n exists and that does not satisfy the conditions of the lemma, 
unstable. A pair (T,p) such that p/A(T) £ Ti n+m and pj n does not exist will be called 
immaterial. 
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Lemma 6.6 There exists an involution X n,m on pairs (T, p) that are unstable. The corre- 
sponding pair (T, p) = T n ' m (T,p) has the property that H^T = H^T, ht n (p) = ht n (p) ± 1 
and type {2 a lb) (T) = type^i b )(T). 



Proof: Begin by setting A = A(H^T) and p = /^(p) then the property that ht n (p) 



ht n (p)±l follows from Lemma l6~2l We want to define T = (H^) _1 H^T, but it is not always 
possible to apply (H^)^ 1 to a tableau. The first m columns of p and p must be the same 
because pi and p\ are both at least n + m and the leftmost m columns of p\ n = p\ n are 
the same as the leftmost m columns of both p and p. Therefore the procedure for applying 
(H^) _1 to the tableau H^T evacuates the labels 12- ••m and hence there is no problem 
defining T in this way. 

This is an involution since /™(p) = p and T = (H^) _1 H^T. 

We also have that H^T = H^T, and hence by Lemma 16. II 

type (2 a lb) (T) = type {2 a lb) (Yl^Yl p m T) = type (2 a l6) (H~ 1 H^f ) = type (2 a lb) (f) (6.6) 



Example 6.7 Let m = 2 and T 



We calculate that H 2 T 



(7,4,1) andf 



2 3 



and p = (8, 3, 1). 



and find that (T, p) is an unstable pair, p 



5 7 



3 5 



Notice that the types of T, T, and H 2 1 ~H 2 T are all the same. type^^^T) 
type(2,2,i)(T) = type (2 ,2,i)( H 2~ lH 2 T ) = (ma, ma, m) . 



Proposition 6.8 

HlH^X;^] = ^^«')( T >iV^')( T ) Sj(T) [I] 

T 
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where the sum is over standard tableau T of size 2a + b + m that contain \l\2\ PA as a 
subtableau and bt m2 a x b )(T) = 6( 2 «if))(H, m 1 T) and 

a (m2 a lb) (T) =c(T) - (2 J - (m - l)(2a + 6) 

a 

- ^((2a + 6 + 1) - 22) X (^e (2alb) (H m 1 T). J = ma) 
i=i 

Remark: For the case that m = 1, #*#( 2 oii.JX; is also Schur positive, but not because 
any of the lemmas in this section hold true and so the following proof does not apply. There 
is an algebraic explanation for this fact. By using commutation relations (jH.lj) and (jH.rij) and 
Theorem 11.21 we have that H\Ht 2 aib\ [X; q, t] = t a H/ 2 aib+i\[X; qt~ x , t]. 

Proof: Set n = 2a + b. For m > 0, the action of H l m on the Schur function basis is given 
in Proposition II. II and using equation (jl.5j) . we have that 

HlH^X-^t] = J2 E (-l)^" 1 ^ 1 *^^^^'^'^.!^ (6-7) 

T p/A(T)GW m +„ 

There is exactly one term in equation (|6.7|) for every pair (T, p) where T is a standard 
tableau of size n and p is a partition such that p/X(T) e Ti, m+n (exactly the pairs for which 
H^Tis defined). The weight of apair (T, p) will be (-l)^(p)- 1 g b (2«i b )( T )t a (2«i")( T ) + l A ^ r l SpJn [X]. 
Note that the immaterial pairs have weight zero. We will use the lemmas above to show 
that the unstable pairs cancel and we will be left with the stable pairs which are in bijection 
with the standard tableau containing 11121 PA as a subtableau. 

If (T, p) is an unstable pair then let (T,p) = T m ' n (T,p). Since the (2 a l b ) — types are 
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the same then we have that bt 2 aib\(T) = br 2a ib\(T) and by Proposition 15.91 and Lemma l6.4| 
a {2 a lb) (T) + \X(T)/p r \=c(T) + \X(T)/p r \ 

a 

- £)((n + 1) - 2i) x {type {2 a lb) {T) i = mo) (6.8) 
i=i 

= c(W m T)-(T)-{m-l)n 

a 

- J2(( n + 1) - 2i)x{tVPH2^){T)i = ma) (6.9) 
= c(H^f) - (2) - (m - l)n 

o 

- + 1) - 2i) X (type {2 a lb) (f) i = m) (6.10) 

i=l 

= c(f ) + |A(f 

a 

- ^((n + 1) - 2i) X (type {2 a lb) (f) i = ma) (6.11) 



i=l 



= o (2 . 1 » ) (f) + |A(f)/p r | (6.12) 

Therefore the unstable pairs (T, p) and (T, p) have the same weight but opposite sign. 

If (T, p) is a stable pair then by Lemma f6. 51 the sign of the term is positive and hence 
are the only terms that contribute to this sum. The map that sends (T, p) to H^T is a 
bijection from these pairs to the standard tableaux that contain 11121 @ as a subtableau. 
Also remark that by Proposition 15.91 and Lemma 16.41 for a tableau of this sort 

a (2al6) (H m 1 T) + \X(T)/p r \ = c(H- 1 T) + \X(T)/p r \ 

a 

- ^((n + 1) - 2i) X (t2/pe (2 „ l6) (H- 1 T) i = ma) (6.13) 
i=i 

= c(T) - (Y) - (m - l)n 

a 

- £)((n + 1) - 20x(ti/pe (2nb) (H m 1 r) J = ma) (6.14) 

i=l 

Hence, the exponent of i in the weight of the pair agrees with our definition of at m2 aib\(T) . 
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Proposition 6.9 

HlH {2alb) [X-q,t\ = Y^q^ 1 ^ T) -y 2 h a ^ {T) s KT) [X] 

T 

where the sum is over standard tableau T of size 2a + b + m that contain 
and for these tableaux, b^ m2 aib^{T) = 6( 2 a 1 b)(H m T) + ( 2 ) and 



m 

2i as a subtableau 
I 



0(m2«i»)(r) = C ( T ) - ^2(( 2a + b + 1) - 2 0x(^pe (2 a 1 t )) (H m 1 T) i = ma) 

8=1 



Proof: All of the arguments and lemmas in this section can be transposed and traced 
through for the case of and H^, but an algebraic method is much easier. Set n = 2a + b. 
We have that H^[X; q,t] = ut n ^q n ^H^[X; l/q, l/t] where n(fi) = - 1)# by [H eq. 
8.14, p. 354], and hence 

HlH^X-.q.t] = uHtfutfH^Xi&t] (6.15) 
= uHVH n t ni2aib U n(a+b ' a) H (2alb) [X-, l/q, l/t] (6.16) 

= t™ + "( 2aib )g a ^t- a (^i b ) ( " T) g-W l6) (^) SA(T) [x] (6.17) 
T 

)«1 b 



^ t n + n(2n^)-a (m2aib) (a,T) ? 6 (m2ai6) (T)-^j s ^ [x] (g lg) 



r 



rere the sum here is over all standard tableaux of size n + m that contain the subtableau 



The t exponent can be reduced after this using Proposition 16.81 Fact 15.51 and the 
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relation a^ 2 a lb ){ujT) = n(2 a l b ) 
n + n {2 a l b ) - a {m2 a lb) (cuT) 



o 

To extend these results to H^i an d H^ 2 , recovering the statistics involves deleting 
the first cell and then reducing to the two column case. We introduce the notation of the 
operator K h that acts on standard tableaux with more than h cells. The result K h T is the 
tableau formed by deleting the cells with labels 1 through h and then lowering the labels in 
the remaining tableau by h. 

Proposition 6.10 

H^H (2 a lb) [X;q,t] = ^ q^^" 1 ^^ s x(T) [X] 

T 

where the sum is over standard tableau T of size 2a + b + 3 that contain as a subtableau. 
For these tableaux set b^ 32 a i b ) 

(T) = 6 (2al6) (H^ 1 ir 1 T) + 1 and 

a 

a [32 a lb) (T) = c(T) - 1 - (2a + b) - ^((2a + b + 1) - 2i) X {ty P e {2alb) (H. 2 1 K 1 T) i = nn) 

i=i 



Proof: 

e 1 [X]H t 2 H {2 a lb) [X;q,t] = ^ q b ^+^ K ' T k a ^ +1 ^ (KlT) s x{T) [X] (6.22) 

T 



a( 2 a 1 6)(T'). 

n{2 a l b ) - c(uT) + (Tj +mn 

a 

+ ^((n + 1) - 2^x(^e (2ali)) (^H m 1 T) i = ma) (6.19) 
i=i 

c(T) - (2) + n(2 a l 6 ) 

a 

+ ^((n + 1) - 2^) X (^e (2ali)) (cuH- 1 T), = nm) (6.20) 
i=i 

a 

c(T) - J]((n + 1) - 20 X (^e (2al6) (H- 1 T) i = m) (6.21) 
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where the sum is over all standard tableaux T that contain either ura or [fk 
For the tableaux that contain maa, c(T) = c{K\T) + 2a + b + 2 and 



a+l 

r/,o„ : Jl6) (^iT) = c(KiT) - J^((2a + b + 3) - 2i) X (type {2a+ i lb) (K 1 T) i = ma) (6.23) 



t( 2 »+il»)l 

i=l 



c(T) - (2a + 6 + 2) - (2a + b + 1] 



a+l 



^((2a + b + 1) - 2z)x(t^e (2Ql6) (H 2 - 1 ^ 1 T) l = ma) (6.24) 



8=1 



For those tableau that have H 3 1 T = H 2 K{I ', it is clear that equation ()6.24j) is equal 
to ar 32 <*ib){T). For the rest we need verify that type^ib^H^ 1 K{T) = type^^itt^T) . 

Let f = H^T so that T = H ^ 3+2a+6+A(T)l ' A(T)r) f . Therefore we have that K{T = 



K' 1 H (3+2a+6+A(T)l ' A(T)r) T = H (2+2a+fe+A(T)l ' A(T)r) f 

type^H^K.T) = ^e (2nb) (H 2 - 1 H( 2+2a+6+A(T)l ' A(T) " ) f ) = type {2alb) (f) (6.25) 
From this we can conclude that 
(e 1 [X]H t 2 - Hl)H (2alb) [X; q,t]=^ g 6 (-+ 1 ^) ( ^ T) t a (-+ 1 1 b )^ lT) SA(T) [X] (6.26) 

T 

where the sum is over all standard tableaux T that contain 0^. For these tableaux we have 
that c(T) = c(KiT) and hence 

a+l 

a (2a+ i l6) (K 1 T) = c^T) - J^((2a + 6 + 3) - 2i)x(type (2a+ i lb) (K 1 T) i = ma) (6.27) 

i=l 

a 

= c(T) - (2a + b + 1) - J^((2a + b + 1) - 2i)x(t^e (2ni , ) (H 2 - 1 fT 1 T) i = ma) 



i=l 



(6.28) 



A similar proof verifies that the operator i/Ha works as expected. 
Proposition 6.11 

H^H {2alb) [X;q,t} = ^gWii> ) ( T )- 2 t a (32«i i >)( T ) SA{T) [x] 
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where the sum is over standard tableau T of size 2a + b + 3 that contain as a subtableau. 
For these tableaux set b( 32 a i b ) 

(T) = 6 (2 a 1 i> ) (H^ 1 T) + 2 and 

a 

a (32alb) (T) = c(T) - 2 - (2a + b) - £((2a + b + 1) - 2*)x(t^e (2al6) (H 2 - 1 K 1 T) l = ma) 

i=i 

These propositions together as well as Theorem 11.31 and equation (j3.7j) show the 
following theorem. 

Theorem 6.12 

H {32alb) [X;q,t] = ^gW>)( T )tW)( T ) SA{T) [X] 

T 

where the sum is over all standard tableaux T of size 3 + 2a + b and a^2 a i h ){T) and b( 32 a i b )(T) 
are defined in Propositions \6. \6.tA \6.1IA and \6.lf\ 

Unfortunately, since there is no general formula for the H s (each one is a spe- 
cial case) and each must be checked separately to show that they have the property that 
H s H^a^lX; q, t) is Schur positive. 

For T a standard tableau of size 2a + b + 4 that contains the standard tableau S of 
size 4 we will define that 

a 

a (42 an)(r) = c(T) - a s - (3 s {2a + b) - £)((2a + 6 + 1) - 2{] X (ty P e {2ali) (e s (T)) i = ra) 

i=i 

(6.29) 

(T) = 6 (2al6) (^(T)) + 75 (6.30) 

where as, /3s, 9s(T), and 7s are given in Table 1. 

The format for the statistics is the same, just a few variables change for each equa- 
tion. It is easier to give a table of values for the parts of this equation that change than 
it is to describe a procedure for calculating them. Below is a table for the statistics for 
H(4 2 aib)[X; q,t] (The values in the table for Hr 32 a i b )[X] Q,t] follow from Propositions 16.81 16.91 
El and EH). 
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S a s (3 S e s {T) 7s H s 



HI3I3I31 


6 


3 


H 4 X T 





hi 




Erbiii 


3 


2 




1 


ei Hi- 


m 


fen 


4 


2 




2 


h 2 H\- 


m 


i 


\m 


5 


2 


H 2 X K 2 T 


3 


h 2 H\- 


h x Ht + Hi 




321 


4 


2 




2 


h 2 H\- 


Hi 




331 


2 


1 


U. 2 X K 2 T 


4 


e 2 H\- 


Hi 




331 


1 


1 


H 2 'K 2 T 


3 


e 2 H\- 


e x H\ + Hi 




331 


2 


1 




A 


p^H 1 — 
z. 2 n 2 


Hi 


[ 


321 




1 


1 1 i -^l J 




Pi H l — 


Hi 




3 


(J 


(J 


tt — It* 

h 4 r 











3 


2 







hi 




E 


331 


2 


1 


H^T 


1 


e x H\- 


m 


E 


321 


1 


1 




2 


eiH\- 


m 














3 


m 





Table 1: Values for variable pieces of formulas ()6.29|) and ()6.30|) 
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Proposition 6.13 For each standard tableau S of size 4 except |jfaa, gj, [?1 we ^o^e 

H s H (2 a lb) [X;q,t] = ^qh^ b ) {T) ^n a ^ b ) {T) s x{T) [X} 

T 

where the sum is over all standard tableaux T of size 4 + 2a + 6 that contain S as a subtableau. 
In addition we have that 

(#ffl + #iW) H {2alb) [X;q,t] = ^ g Vi 6 )( T )- 2 iVi')( T ' SA(T) [l] 

T 

where the sum is over all standard tableaux T of size 4 + 2a + 6 that contain either or H3 . 
Furthermore, 

nM + lMa\ H (2alb) [X;q,t} = ^ g W)( T )-V(^) (T) s A(T) [X] 
/ T 

where the sum is over all standard tableaux T of size 4 + 2a + b that contain either m or m^ . 

The fact that Hu2 a i b )[X] q, t] is Schur positive follows immediately from this propo- 
sition and Theorem II .41 and equation (|4.9jl . 

Theorem 6.14 

H {42alb) [X;q,t] = ^/<« 4 ) (T) tV' ^ T) sx { t)[X} 

T 

where the sum is over all standard tableaux T of size 4 + 2a + b. 



Proof: fof Proposition IB. 13|) 

We note that if T contains rnmrn or f, it is not necessarily the case that H4T = 
H^KiT = H 2 1 K 2 T, but it will be true that type^^-jiH^T) = type ^^(ii^ K\T) = 
tyP e (2 a i b )(H.2 1 K2T) . Similarly, if T contains ^ or then in general it is not true that 
K±T = 'H 2 1 K 2 T^ but it will be true that type^a^ili^ 1 K\T) = type^ 2al b^{Yl 2 l K 2 T). 
This follows from a similar argument to the one used in equation ()6.25|) . 

For each operator H s we must verify that it has the property stated in the propostion. 

HI 

The case of if 1 ' 2 ' 11141 and IM follow from Proposition 16.81 and 16.91 respectively. 
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Since H\Hi 2a ,xb\ [X\ q, t] is a generating function for the standard tableaux that contain 
ima then ei[X]H^Hf 2 aib^[X; q, t] is a generating function that for the standard tableaux that 
contain or [jbrai , that is 

ei [X]#*# (2n6) [X; = ^ g 6 (2n^ H 3" 1 ^ T ) t "(3 2 <.^)(^ T ) SMT) [l] ( 6 . 3 l) 

T 

To verify this proposition in this case we need to show that a^ 32 a i b ){^iT) = a^a^^T) 
for these tableaux. For the tableaux that contain g]^, this is true by definition. For the 
tableaux that contain 11 we note that c(KiT) = c(T) — (2a + b + 3) and type^a^CH^T) = 
type (2 aif. ) (Hj 1 KiT). 

fflrS _|_ jj jfbra - 

h 2 H 2 H( 2 a i b )[X; 9; is a generating function for the standard tableaux that contain 
El, Bn or niwi as subtableau. 

T 

We verify for these standard tableaux that a^ 2 a+iib^(K 2 T) = a^ 42 a \ b ){T) For the tableaux 
that contain g§] or g]^ as standard subtableau we have that c(K 2 T) = c(T) — (2a + 6 + 3). 
For the tableaux that contain m, c(K 2 T) = c(T) — (2(2a + 6) + 5) and we note that 
typepaib^H^KxT) = type {2alb) {H 2 1 K 2 T). 



e 2 -f^2-^(2 a i i, )[^; <?5 1] is a generating function for the standard tableaux that contain 
and [flam as a subtableau with the formula 

h 2 [X\HlH {2alb) [X- q,t] = ^gVi")^ 1 ^^^ 1 !")^ 5 ")^^)^] (6.33) 



The operator ffti = e 2 H 2 — i/fflaa. We calculate that for the standard tableaux that contain 
c(K 2 T) = c(T) and for the standard tableaux the contain Hba , c(K 2 T) = c(T) — (2a+6+2). 

The verification of the cases for the operators with the transpose tableaux is nearly 
identical or an algebraic argument can be used, as = 6— a u s an d Ps — 3— /J^s for all standard 

tableaux S of size 4. Let n = 2a + b. It follows from the relation H s = ujH 1 ^' 3 ujR 1 and 
t-^i/t 

equation (jdl8|) that 
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H" s H (2 « lb) [X- q,t] = ^ t «+«(2 a i 6 )-« ( 4 2aii >)(^) g «(«+M-fe (42ai6) (-T) SA(r)[x] (6 _ 34) 

T 

where the sum is over all standard tableaux T that contain uS. 
Now for the statistic, we have that 

n + n(2 a l h ) - a (42 a lb) (ujT) =n + n(2 a l b ) - {c(uT) - a s - Psn (6.35) 

a 

- ^((2a + b + 1) - 2i)x{type (2alb) (0 s {ujT)) i = ma)) 

i=l 

= c(T) - (n + 4)(n + 3)/2 + n(n - l)/2 + n + a s + /3<?n (6.36) 

a 

- J^((2a + 6 + 1) - 2i) X (type i2 a lb) (e u , s (T)) i = ma) 

i=l 

= c(T) - (3 - /3 s )n - (6 - a s ) 

a 

- ^((2a + 6 + 1) - 2i)x(type i2 a lb) (eu S (T))i = ma) 

i=l 

= a ( -4 2 a 1 6)(T) 

o 



It makes sense here to generalize the notion of the type^a^ as it was introduced in 
[22] to the case of (32 a l 6 ) and (42 a l b ). For m = 3 or 4, T be a standard tableau of size 
m + 2a + 6 that contains the standard tableau S of size m. Define the type^ m2 a lb -j(T) = 
(S,type(2*ib)(0s(T))) where Os{T) is given in Table 1. 

We finish this article by noting that the following analog to Conjecture 4.8 of [22] 
seems to be true by experimental calculation. 

Conjecture 6.15 The number of tableaux of a fixed (m2 a l b ) —type, (S,s), and fixed a( m 2«i b ) 
value, i, increases to a maximum and then decreases as i ranges over all possible values. 

More precisely stated, if S is a standard tableau of size m (m =3 or 4) and s is 
a sequence of a standard tableaux of size 2 and b of size 1, then let A % , Sg \ = #{T|T G 

ST m+2a+ \type {m2alb) {T) = {S, s), a (m2 a lb) {T) = i}. The sequence A\ S s) = (a ^ A\ s>s) , Af^, 
is unimodal. 
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Example 6.16 Let s = (m,m,m) and let S range over all standard tableaux of size 3. 



S = DEE] 


A* 


= (1,2,3,4,2,1,1) 


S = Irbi 


A* 


= (2,4,6,5,4,2,1) 


S = Irfai 


A* 


= (1,2,4,5,6,4,2) 




A* 


= (1,1,2,4,3,2,1) 



Let s = (m,[n) and let S range over all standard tableaux of size 4. This partitions all 
of the standard tableaux of size 6 into 10 different types. 



R = wmu\ 


A* 


= (1,2,1,1) 


S — Irbrai 


A* 

A (S,s) 


= (2,4,2,1) 


S = fflai 


A* 


= (2,4,2,1) 


S = [ffarei 


A* 


= (1,2,3,3) 




A* 


= (1,2,2,1) 


C — EE] 


A* 


= (1,2,2,1) 




A* 

A (S,s) 


= (3,3,2,1) 


S = h 


A* 


= (1,2,4,2) 




A* 


= (1,2,4,2) 


s = i 
i 


A* 


= (1,1,2,1) 



Example 6.17 Let s = (ma, ma, m) and S range over all standard tableaux of size 3. 



S = H12151 


A ts, s) 


= (1,4,6,8,7,6,4,2,1,1) 


S — Irbi 


A* 


= (2,3,8,12,13,10,8,4,2,1) 


S = [fbi 


A* 


= (0,0,2,7,12,14,13,9,4,2) 




A ts,s) 


= (0,0,1,3,5,7,7,4,2,1) 
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